Cognition-Oriented Building Blocks of Future Benchmark Scenarios for Humanoid Home Robots

Comparing intelligent robots interacting with humans is a difficult task. First of all, robotic helps in everyday life can be involved in different tasks. Additionally, the grade of autonomy of the robot and the type of interaction between human and robot may differ to a great extent. It also is an intricate task to design common metrics to assess human-robot interaction. Thus normal benchmarks fail in this case. After an overview of already existing benchmarks in robotics this paper sketches affordances for benchmarks for human-robot interaction in a home environment. Many aspects of already existing benchmarks, evaluation criteria, and metrics have to be combined to asses human-robot interaction in a complex environment. The great difficulty is measuring human and robot behavior; here novel approaches taking into account perspectives from different fields (i.e. engineers, psychologists, sociologist) are needed, as a human being does not behave as anticipated by the system designer most of the time.